Method for detecting movement of image sensors

ABSTRACT

A method to detect the movement of an image sensor according to captured images. An image region is captured from a first image. Then, a first corresponding region matching the first region is captured from a second image. A second image region is captured from the second image. Then, a second corresponding region matching the second captured image region is captured from the first image. Finally, the movement of the image sensor is determined according to the first region and the first corresponding region when a first relative distance between the first region and the first corresponding region is the same as a second relative distance of the second captured image region and the second corresponding region, but in the opposite direction.

This application claims priority from Taiwanese application no.90122445, filed in Taiwan, R.O.C., on Sep. 11, 2001, pursuant to 35U.S.C. 119(a)–(d).

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates in general to a method to detect themovement of an image sensor. In particular, the present inventionrelates to a method for detecting movement of an image sensor which, bycomparing the images captured by the image sensor at differentlocations, finding the matched images and performing some verificationprocesses, the movement of the image sensor is determined.

2. Description of the Related Art

Normally, the prior arts use block matching to determine the movement ofan image sensor. Ideally, using this method can correctly find a matchedframe block generated from the matching function, mean squared error(MSE) or mean absolute difference (MAD), then use the matched frameblocks to calculate the movement of the image sensor.

FIG. 1A and FIG. 1B show the conventional method to determine themovement of an image sensor by block matching. The frame 10A is theimage captured by the image sensor in a first location, and the frame10B is the image captured by the image sensor when the image sensormoves to a second location. Here, the size of the frame captured by theimage sensor is 6×6 units, and the frame 10A is divided into 9 parts by2×2 units. Then, the frame 10B is searched by block matching to find aframe block matching the frame block 12A in frame 10A.

However, there are actually many sources of noise, not only from imageand process variation, but also from the environment of the image sensorsuch as temperature and illumination variation. Therefore, the matchingresult can only obtains a frame block 12B having a gray scale close tothe frame block 12A, wherein the value of the matching function is theminimum. Then, a frame block 14B matching the frame block 14A isobtained from the image 10B by the same way.

In addition, the prior art has to compare all the frame blocks in theoriginal images to determine the movement of the image sensor accordingto the frame 10A and 10B, that is, the conventional method has to fullysearch to get the result. Therefore, the conventional method requiresusing a large amount of data, so the determining rate is slow.

SUMMARY OF THE INVENTION

The object of the present invention is to provide a method to detect themovement of an image sensor. When comparing the original image and thefollowing image, the present invention doesn't compare all the frameblocks in the original image with the following image. Rather, itselects a single frame block in the original image to fully search thefollowing image. In addition, the present invention provides threeverification methods to ensure the accuracy of the comparison. The threeverification methods are cross checking, maximum error threshold, anddetermining the matched frame block number of the result of thecomparison. The present invention quickly and accurately determines themovement of an image sensor using less data and operation.

To achieve the above-mentioned object, the present invention provides amethod to detect the movement of an image sensor according to thecaptured images, including the following steps. An image region iscaptured from a first image. Then, a first corresponding region matchingthe first image region is captured from a second image. A second imageregion is captured from the second image. Then, a second correspondingregion matching the second image region is captured from the firstimage. Finally, the movement of the image sensor is determined accordingto the first region and the first corresponding region when a firstrelative distance between the first region and the first correspondingregion is the same as a second relative distance of the second capturedimage region and the second corresponding region, but in the oppositedirection.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will become more fully understood from thedetailed description given hereinbelow and the accompanying drawings,given by way of illustration only and thus not intended to be limitativeof the present invention.

FIG. 1A and FIG. 1B show the conventional method to determine themovement of an image sensor by block matching.

FIG. 2A and FIG. 2B show the method to determine the movement of animage sensor by block matching according to the embodiment of thepresent invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

First Embodiment

FIG. 2A and FIG. 2B show the method to determine the movement of animage sensor by block matching according to the embodiment of thepresent invention.

The first frame 20A is the image captured by the image sensor in a firstlocation, and the second frame 20B is the image captured by the imagesensor when the image sensor moves to a second location. Here, the sizeof the frame captured by the image sensor is 6×6 units. The frame block22A is captured from the first frame 20A, wherein the size of the frameblock 22A is 2×2 units. Then, frame block matching is performed to findthe frame block 22B matching the frame block 22A in the second frame20B, wherein the value of the matching function is the minimum. Inaddition, the frame block 22A is near the center of the first image 20Ato avoid missing the matched frame block in the second image because theimage sensor has moved too far.

Next, a verification process is performed, the present embodimentperforming cross checking to verify the determination. The crosschecking catches another frame block 24B from the center of the secondimage 20B and fully searches to find a matched frame block 24A in thefirst image 20A according to the frame block 24B, then calculates thedislocationment and moving direction of the frame blocks 22B and 22A(represented as the vector 26A) and the dislocationment and movingdirection of the frame blocks 24B and 24A (represented as the vector26B). The movement and moving direction of the image sensor are detectedaccording to the frame blocks 22B and 22A when the vectors 26A and 26Bhave the same length and opposite directions.

Therefore, the movement and moving direction of the image sensor isdetected with less data use.

Second Embodiment

FIG. 2A and FIG. 2B show the method to determine the movement of animage sensor by block matching according to the embodiment of thepresent invention.

The first frame 20A is the image captured by the image sensor in a firstlocation, and the second frame 20B is the image captured by the imagesensor when the image sensor moves to a second location. Here, the sizeof the frame captured by the image sensor is 6×6 units. The frame block22A is captured from the first frame 20A, wherein the size of the frameblock 22A is 2×2 units. Then, frame block matching is performed to findthe frame block 22B matching the frame block 22A in the second frame20B, wherein the value of the matching function is the minimum. Inaddition, the frame block 22A is near the center of the first image 20Ato avoid missing the matched frame block in the second image because theimage sensor has moved too far.

Next, a verification process is performed, the present embodimentperforming the maximum error threshold method to verify thedetermination, which skips the search result when the value of thematching function by comparing the frame block 22B and frame block 22Ais larger than a maximum error threshold value because the search resultis wrong, which will cause errors in operation.

In this embodiment, if the image sensor has 6 bits resolution and 6×6unit size, and assuming the tolerable noise is 10%, the maximum errorthreshold value is 6.4 (2^6×10%), which is definded by mean absolutedifference.

Therefore, more correct movement and moving direction of the imagesensor is detected.

Third Embodiment

FIG. 2A and FIG. 2B show the method to determine the movement of animage sensor by block matching according to the embodiment of thepresent invention.

The first frame 20A is the image captured by the image sensor in a firstlocation, and the second frame 20B is the image captured by the imagesensor when the image sensor moves to a second location. Here, the sizeof the frame captured by the image sensor is 6×6 units. The frame block22A is captured from the first frame 20A, wherein the size of the frameblock 22A is 2×2 units. Then, frame block matching is performed to findthe frame block 22B matching the frame block 22A in the second frame20B, wherein the value of the matching function is the minimum. Inaddition, the frame block 22A is near the center of the first image 20Ato avoid missing the matched frame block in the second image because theimage sensor has moved too far.

Next, a verification process is performed, the present embodimentascertaining the result of the comparison by determining the number ofthe matched frame block. Therefore, if the number of the frame blocks inthe second image 20B matching the frame block 22A is more than one, thesearch result is skipped because the search result is wrong, which willcause errors in operation.

Although embodiments of the present invention skip data which might bewrong, there is no influence on the whole determination result of theimage sensor because the frame rate of the image sensor is very high,therefore, user viewability is not affected. As for an optical mouse,although skipping some wrong data causes mark stop, the mark goes to thecorrect location according to the next frame. However, if the mark ofthe optical mouse is moved according to the wrong data, the mark will goto the location unexpectly, which confuses the user operating theoptical mouse.

Accordingly, the method according to the present invention employscapturing a part of frames and combining the verification methodsmentioned above to obtain the movement of an image sensor, whichdecreases the data and simplifies the determination.

The foregoing description of the preferred embodiments of this inventionhas been presented for purposes of illustration and description. Obviousmodifications or variations are possible in light of the above teaching.The embodiments were chosen and described to provide the bestillustration of the principles of this invention and its practicalapplication to thereby enable those skilled in the art to utilize theinvention in various embodiments and with various modifications as aresuited to the particular use contemplated. All such modifications andvariations are within the scope of the present invention as determinedby the appended claims when interpreted in accordance with the breadthto which they are fairly, legally, and equitably entitled.

1. A method for detecting movement of an image sensor, during the movement of the image sensor, the image sensor capturing a first image at a first location and subsequently capturing a second image at a second location, the method comprising the following steps: defining a first image region within the first image; locating a first corresponding region corresponding to the first image region within the second image; determining a first relative distance between the first image region and the first corresponding region; defining a second image region within the second image; locating a second corresponding region corresponding to the second image region within the first image; determining a second relative distance between the second image region and the second corresponding region; and accumulating the movement of the image sensor according to the first relative and second relative distances.
 2. The method for detecting movement of an image sensor as claimed in claim 1, wherein the first image region is substantially centered within the first image.
 3. A method for detecting movement of an image sensor, during the movement of the image sensor, the image sensor capturing a first image at a first location and subsequently capturing a second image at a second location, the method comprising the following steps: defining a first image region substantially centered within the first image; comparing the first image region with more than one image region within the second image to locate a first corresponding region matching the first image region within the second image; accumulating the movement of the image sensor according to the first image region and the first corresponding region; determining a first relative distance between the first image region and the first corresponding region; defining a second image region within the second image; locating a second corresponding region corresponding to the second image region within the first image; determining a second relative distance between the second image region and the second corresponding region; and accumulating the movement of the image sensor according to the first relative and second relative distances. 